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Detailed Action 

1. This office action is in response to Amendment B filed on 05/17/01 regarding U.S. 
Application. No. 09/396,701, where claims 1-6, 8-25, 27-33 and 35-42 remain pending. 

\ 

2. Claim 39 is objected to because of the following informalities, dependent claim (39) 
makes reference to limitation on independent claim (37), claim (39) recites: "as recited in claim 
37, wherein the extracting annotation identification information comprises extracting from the 
email message:".... However amended claim 37, no longer comprises extraction features made 
reference to in claim 39, due to amended B. Appropriate correction is required. 

Claim Rejections - 35 USC §103 

3. Quotation of 35 U.S.C. §103(a) which forms the basis for all obviousness rejections set 
forth in this Office action may be found in previous office action: 

4. Claims 1-5, 13-16, 27-28, 3 1-32 and 37-39 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hou et. al. (Hou) U.S. Patent No. 5,838,313 in view of Sidana U.S. Patent No. 
6,081,829 in further view of Pizano et. al. (Pizano) U.S. Patent No. 6,105,055. 

Regarding claim 1-3, and 29-30, Hou teaches substantial features of the invention as claimed; 
teaching a system comprising: a client computer (col 6/lines 38-51) to playback multimedia 
content (col 8/lines 1-15, col 6/lines 30-37) and annotations corresponding to different temporal 
portions of the multimedia content (col 6/lines 61-66, col 7/lines 1-3, col 8/lines 61-63, col 
9/lines 9-14); maintaining an annotation database (Fig. 9, 33, col 6/lines 12-14) having a 
plurality of annotations (col 3/lines 54-col 4/line 25) corresponding to the multimedia content 
(col 6/lines 38-41), provide the plurality of annotations to the client computer for playback (Fig. 
6, playback means, Fig. 12, lines 56-61, col 6/lines 23-25, display annotations means, col 8/lines 
43-48), send electronic mail messages (col 3/lines 8-13, create or reply means: col 2/lines 25-37, 
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sending and displaying mail: col 11/lines 22-25, creating e-mail message: col 6/lines 14-16) 
including annotations (col 11/lines 3-25, Fig. 1) to recipients identified by the client computer 
(col 4/lines 26-31, Fig. 4), via interface means to receive input data regarding new annotation 
corresponding to media content (col 6/lines 26-51), generate new annotations create or reply 
means (col 2/lines 25-37, col 3/lines 9-13) based on received electronic mail messages (col 
4/lines 10-14, Fig. 2), and add the new annotations (col 9/lines 15-32, col 1 /lines 17-20, adding 
annotations means: col 5/lines 29-52, report: col 6/lines 10-14, Fig. 3, saving in database) to the 
annotation database, wherein generate new annotation based on electronic mail message received 
both in response to the sent electronic mail messages and not in response to the sent electronic 
mail messages (Hou: col 2/lines 25-30, col 50-col 3/line 13, col 3/line 65-col 4/line 14, col 
4/lines 26-3 1, col 6/lines 14-16), identifier of a temporal range of the media content that the new 
annotation content is associated with, which corresponds to a location of the media content that 
is after the beginning of the media content and at which rendering of the media content should 
begin in response to selection of the identifier of the media content in the electronic mail 
message; (Hou: col 6/lines 61-66, col 7/lines 1-3, col 8/line 61-63, col 9/lines), wherein said 
identifier of a temporal segment of the multimedia content is after the beginning of the 
multimedia content; disclosing means for identifying temporal ranges or segment of multimedia 
content with which the new annotation is associated, such as time temporal range denoting the a 
segment of time (e.g. beginning/end), temporal range of information denoting the content, range, 
category type, beginning and end of a segmented multimedia content supporting means to store, 
retrieve, render multimedia content, annotation sets corresponding to the multimedia categories 
(e.g. image, text, audio), that the new/stored annotation is part of (Hou: Figs. 6, and 14-15, col 
7/lines 4-51, col 2/lines 52-61, col 6/lines 10-14), further teaching an identifier of the media 
content to which the new annotation corresponds, wherein generated electronic message was not 
received in reply to a previous electronic mail message that included annotation (Hou: generation 
of an e-mail message containing annotation, where generated e-mail message is not in response 
to a sent message, col 3/lines 8-13, generation of new report, where report was not a stored 
previously in a mailbox, col 3/lines 65-col 4/line 14, multimedia report 32 generates an e-mail 
message for delivery, col 6/lines 4-25, generation of an e-mail message containing annotation, 
where generated e-mail message is not in response to a sent message); 
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However Hou does not explicitly teach analyze said electronic mail message to locate 
data in the electronic mail message, generate new annotation that included the located data, and 
add the new annotation to a annotation database, and maintaining said annotation database 
having a plurality of annotations corresponding to multimedia content; 

Sidana teaches a system/method for enabling client using an user interface to be 
presented with web-viewable documents including annotations maintaining a annotation 
multimedia server computer coupled to the client for streaming the multimedia to the client 
computer (abstract, col 2/lines 34-59), disclosing a system wherein a client computer is provided 
with annotations corresponding to multimedia content, where annotation media server coupled to 
the client computer is coupled to an annotation server via a network (col 4/lines 14-33, Fig. 1, 
element 106, 1 10, 130, annotation server 120), where an identifier, identifiers the media content 
associated with new annotation content (col 8/lines 5-12); 

however Hou nor Sidana does not explicitly teach means analyzing said electronic mail 
message to locate data in the electronic mail message, generate new annotation that included the 
located data, and add the new annotation to an annotation database further maintaining said 
annotation database having a plurality of annotations corresponding to multimedia content; 

Pizano teaches system/method having means for creating an electronic mail message 
comprising multimedia content, system/method having means for processing created electronic 
mail messages and locating data in the electronic mail message, and add a new annotation to a 
annotation (24) database and further maintaining said annotation database having a plurality of 
annotations corresponding to multimedia content (Pizano, col 2/lines 32-60, email messages, 
containing multimedia (voice/graphic) data, col 1 /lines 60-67, containing dynamic annotations, 
col 3/lines 32-34, creating a new/existing electronic mail messages means, col 3/lines 43-47, 
stored in database identified data in electronic mail message and storing in said database, 
generating a new annotation to said storing database and maintaining said annotation database, 
(col 5/lines 26-36); 

It would have been obvious to one ordinary skilled in the art at the time the invention was 
made to modify Hou's system with means where an annotation server, coupled to the client 
computer to maintain an annotation database and to manage streaming the multimedia content to 
the client computer, as taught by Sidana, motivation would be enable the user via multimedia 
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content identifiers to select for viewing or not a annotation included in a document, access a 
media server to obtain multimedia content corresponding to the content identifier which may 
further be presented in hierarchical for, where and annotation set identifier one or more sets of 
annotations. Further modify existing system with means for analyzing said electronic mail 
message to locate data in the electronic mail message, generate new annotation that included the 
located data, and add the new annotation to a annotation database further maintaining said 
annotation database having a plurality of annotations corresponding to multimedia content, as 
taught by Pizano, motivation would be to use off-shelf components that supports information 
sharing and distribution advancing system with document conferencing functions that enables 
multiple users simultaneously view and annotate multimedia data, complementing existing 
electronic mail messaging annotation capabilities. 

Regarding claim 4, the combined teachings as discussed above, wherein the client computer is 
further present an electronic mail message including a multimedia content identifier to a user 
(Hou: annotations identifiers col 6/lines 61-66, col 7/lines 1-3, col 8/lines 61-63, col 9/lines 9- 
14), receive a user selection of the multimedia content identifier, access a media server to obtain 
the multimedia content (Sidana: col 7/lines 62-col 8/line 26, Hou: col 11/lines 26-34, playback 
the multimedia content to the user). 

Regarding claim 5, the combined teachings as discussed above, further teach means for 
transmitting to the media server, an identifier of a temporal segment of the multimedia content, 
and wherein the media server is to stream to the client computer the multimedia content 
beginning with the identified temporal segment (Hou: col 6/lines 61-66, col 7/lines 1-3, col 8/line 
61-63, col 9/lines), wherein said identifier of a temporal segment of the multimedia content is 
after the beginning of the multimedia content; disclosing means for identifying temporal ranges 
or segment of multimedia content with which the new annotation is associated, such as time 
temporal range denoting the a segment of time (i.e. beginning/end), temporal range of 
information denoting the content, range, category type, beginning and end of a segmented 
multimedia content supporting means to store, retrieve, render multimedia content (e.g. 
load/display, time, begin/end of objects), annotation sets corresponding to the multimedia 
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categories (e.g. image, text, audio), that the new or stored annotation is part of (Hou: Figs. 6, 14- 
15, col 7/lines 4-51, col 2/lines 52-61, col 6/lines 10-14, col 7/line 4/line 51). 

Regarding claim 13, and 16, the combined teachings as discussed above, wherein the 
method/program (Sidana: col 1/lines 55-62) further teach means for receiving data for a new 
annotation corresponding to a temporal range (Hou: col 6/line 61-66, col 7/lines 1-3, col 8/lines 
61-63, col 9/lines 9-14) of media content (Hou: col 1/lines 39-44, col 3/lines 9-13); generating an 
electronic mail message including both the content of the new annotation and an identifier of the 
media content (Hou: create electronic mail messages col 3/lines 8-13, create means: col 2/lines 
25-37, col 3/lines 9-13, via interface means to receive input data regarding new annotation 
corresponding to media content col 6/lines 26-51, containing media content identifiers such as 
temporal range markers col 6/line 61-66, col 7/lines 1-3, col 8/lines 61-63, col 9/lines 9-14, col 
1/lines 39-44); and forwarding the electronic mail message to a recipient identified by the data 
(Hou: sending/displaying mail: col 1 1/lines 22-25, col 6/lines 14-16 including annotations col 
1 1/lines 3-25, Fig. 1). 

Regarding claim 14, the combined teachings as discussed above, further teach the generating 
further comprises including, in the electronic mail message, an identifier of the new annotation 
(Sidana: col 8/lines 5-12). 

Regarding claim 15, the combined teachings as discussed above, further teach the generating 
further comprises including, in the electronic mail message, an identifier of the temporal range of 
media content (Hou: col 8/lines 61-63, col 9/lines 1-4, col 6/lines 61-66, col 7/lines 1-3). 

Regarding claim 27, the combined teachings as discussed above, further teach the electronic mail 
message further includes a unique identifier of the new annotation (Sidana: col 7/lines 62-col 
8/line 26). 

Regarding claim 28, the combined teachings as discussed above, further teach wherein the 
electronic mail message further includes an identifier of one or more of a plurality of annotation 
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sets, corresponding to categories for annotations, that the new annotation is associated with 
(Hou: col 5/lines 10-28, Figs. 6, 14-15, col 7/lines 4-51, col 2/lines 52-61, col 6/lines 10-14, col 
7/line4/line51). 

Regarding claim 31, the combined teachings as discussed above, further teach wherein a method 
comprising: receiving an electronic mail notification of an annotation corresponding to media 
content (Hou: col 6/line 61-66, col 7/lines 1-3, col 8/lines 61-63, col Mines 9-14) of media 
content col 1/lines 39-44, col 3/lines 9-13, receiving an electronic mail message including both 
the content of the new annotation and an identifier of the media content create electronic mail 
messages col 3/lines 8-13, col 2/lines 25-37, containing media content identifiers/temporal range 
markers col 6/line 61-66, col 7/lines 1-3, col 8/lines 61-63, col 9/lines 9-14, col 1/lines 39-44); 
and replying to the electronic mail notification to generate a new annotation corresponding to the 
media content (Hou: reply means: col 2/lines 31-37), including, in the reply, an identifier of one 
annotation sets of set of plurality of annotation sets that the new annotation is associated with; 
(Sidana: col 7/lines 62-col 8/line 26, Hou: col 5/lines 10-28, Figs. 6, 14-15, col 7/lines 1-51, col 
2/lines 25-37, 52-61, col 6/lines 10-66, col 3/lines 8-13, col 8/lines 61-63, col 9/lines 9-14, col 
1/lines 39-44, col 1 1/lines 3-25, Fig. 1), displaying a default, corresponding to the identified one 
or more annotation sets, that is to receive and electronic mail notification of the new annotation; 
as discussed above, Hou: col 4/lines 26-31, Fig. 4); wherein the annotations sets corresponds to 
categories for annotations; (Hou: different types of media, static annotation means allows the 
user to add drawings and text to be sent via email, allows the user to record/playback 
annotations, an use can create reports by including multiple media data; col 2/line 50-col 3/line 
13); 

Regarding claim 32, the combined teachings as discussed above, further teach wherein the 
replying includes: obtaining, from the electronic mail notification, as discussed above, including 
an identifier of the annotation; and including the identifier of the annotation in the reply (Hou: 
col 5/lines 10-28). 



Application/Control Number: 09/396,701 
Art Unit: 2152 



Regarding claim 37, the combined teachings as discussed above, further teach one or more 
computer-readable media having stored thereon a computer program that, when executed by one 
or more processors, causes the one or more processors to perform functions including: receiving 
an electronic mail message; and extracting both annotation content and annotation identification 
information from the electronic mail message (Hou: col 1 1 /lines 26-34), wherein generate new 
annotation based on electronic mail message received both in response to the sent electronic mail 
messages and not in response to the sent electronic mail messages (Hou: col 2/lines 25-30, col 
50-col 3/line 13, col 3/line 65-col 4/line 14, col 4/lines 26-31, col 6/lines 14-16); analyze said 
electronic mail message to locate data in the electronic mail message, generate new annotation 
that included the located data, and add the new annotation to a annotation database, further 
adding/maintaining said annotation database having a plurality of annotations corresponding to 
multimedia content; (Sidana, abstract, col 2/lines 34-59, col 4/lines 14-33, Fig. 1, elements 106, 
110, 130, and 120, identifiers the media content associated with new annotation content, col 
8/lines 5-12; Pizano; col 2/lines 32-60, email messages containing multimedia data, col 1/lines 
60-67, containing dynamic annotations, col 3/lines 32-47, stored in database identified data in 
electronic mail message and storing in said database, generating a new annotation to said storing 
database and maintaining said annotation database col 5/lines 26-36); 

Regarding claim 38, the combined teachings as discussed above, further teach wherein the 
computer program further causes the one or more processors to perform functions including: 
creating a new annotation based on the extracted annotation content and the annotation 
identification information (Hou: col 5/lines 5-28); and adding the new annotation to an 
annotation database (Hou: add the new annotations col 9/lines 15-32, col 1/lines 17-20, adding 
annotations means: col 5/lines 29-52, report: col 6/lines 10-14, Fig. 3, saving in database) to the 
annotation database, Sidana software program implementation: col 1/lines 55-62). 

Regarding claim 39, the combined teachings as discussed above, further teach wherein the 
extracting annotation identification information comprises extracting from the email message: an 
identifier of media content to which the annotation content corresponds and of a temporal 
segment (Hou: identifier of a temporal segment of the multimedia content, and wherein the 
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media server is to stream to the client computer the multimedia content beginning with the 
identified temporal segment, col 6/lines 61-66, col 7/lines 1-3, col 8/line 61-63, col 9/lines), 
corresponding to the annotation content, of the media content; and an identifier of an annotation 
set that a new annotation including the extracted annotation content is to be part of (Hou: col 
5/lines 5-28), wherein the annotation set is one of one or more annotation sets corresponding to 
categories for annotations (Hou: Figs. 6, 14-15, col 7/lines 4-51, col 2/lines 52-61, col 6/lines 10- 
14, col7/line4/line51). 

5. Claims 6, 8-13, 17-25 33, 35-36 and 40-42 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hou-Sidana-Pizano in view of Birrel et. al. (Birrel) U.S. Patent No. 6,009,462 
in further view of Russell et. al. (Russell) U.S. Patent No. 5,526,407. 



Regarding claims 6, 13, and 25 the combined teachings of Hou and Sidana as discussed above, 
method comprising presenting, to a user, a user interface (Hou: Fig. 5, col 2/lines 62-col 3/line 
14, canvas, col 4/lines 32-col 5/lines) allowing the user to create a new annotation (Hou: col 
2/lines 52-56, adding/generating means: col 6/lines 10-37) corresponding to media content (Hou: 
Fig. 5); and including, as part of the user interface (Hou: abstract: col 1/lines 65-col 2/line 5), a 
field via which the user can identify a recipient that is to receive an electronic mail notification 
of the new annotation (Hou: col 4/lines 26-32, Figs. 4, 12, Sidana: col 1/lines 55-62), and 
wherein generate new annotation based on electronic mail message received both in response to 
the sent electronic mail messages and communicated as an electronic message that is not in 
response to the another electronic mail messages (Hou: col 2/lines 25-30, col 50-col 3/line 13, 
col 3/line 65-col 4/line 14, col 4/lines 26-31, col 6/lines 14-16), the new annotation to annotation 
server (see Sidana: abstract, col 2/lines 34-59, col 4/lines 14-33, Fig. 1, (106, 110, 130), 
annotation server 120, col 2/lines 41-59, col 4/lines 20-31, where an identifier, identifiers the 
media content associated with new annotation content (see Sidana: col 8/lines 5-12); wherein 
generated/communicated electronic message was not received in reply to a previous electronic 
mail message that included annotation (Hou: generation of an e-mail message containing 
annotation, where generated e-mail message is not in response to a sent message, col 3/lines 8- 
13, generation of new report, where report was not a stored previously in a mailbox, col 3/lines 
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65 -col 4/line 14, multimedia report generates an e-mail message for delivery, col 6/lines 4-25, 
generation of an e-mail message containing annotation, where generated e-mail message is not in 
response to a sent message); 

However the above combined teachings do not explicitly teach where field via which the 
user can identify a recipient that is to receive an electronic mail of the new annotation consisting 
of particularly an electronic mail notification; wherein said communicated electronic mail 
message includes a new annotation corresponding to media content created by a user and an 
identifier of the media content but does not include the media content, wherein identifier of the 
media content to which the new annotation corresponds includes a particularly an user-selectable 
link to said media content. 

Birrel teaches means for composing/accessing an electronic mail message (Birrel, 
composing/accessing an email message, col 1/lines 38-50, composing email means, col 4/lines 
31-50), includes a new annotation corresponding to media content (Birrel, composition means 
enable multimedia content, col 4/lines 59-61) created by a user using a user interface (Birrel, col 
13/lines 9-14, using interface 115, col 2/line 62-col 3/line 8) and an identifier of the media 
content but does not include the media content; (Birrel; electronic mail message includes an 
annotation (e.g. reference) corresponding to the media content created by the user and an 
identifier, said identifier associated with reference, col 12/lines 2-27 wherein said identifier of 
the media content to which the new annotation corresponds includes a user-selectable link to said 
media content used to retrieve multimedia content at location associated with the identifier upon 
user selection of the media content associated with said annotation, wherein the said annotation 
does not include the media content created by the user, col 12/lines 59-col 1 3/line 13); 

Official Notice (see MPEP § 2144.03 Reliance on "Well Known" Prior Art) is 
taken that the following email features was old and well known in the Data Processing art. It 
would have been obvious to one of ordinary skill in the art at the time of applicant's invention to 
include email application having means for (i) a field of the interface user via which the user can 
identify a recipient(s) that is to receive an electronic mail notification (ii) receive an electronic 
mail thread including a plurality of electronic mail messages (iii) displaying a electronic mail 
message notification that includes an identified annotation set (e.g. attachment) that is received 
within an electronic mail message (e.g. inbox window); motivation would be to include a Names 
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dialog box from which a user can choose the recipients and cal also add recipients to the Cc 
field, which sends a copy of the message to its addresses, and add deliver options group, 
enabling user to delay mailing to selected recipients selected via recipient fields until certain 
date, or an notification option that notifies user when sent message has been delivered and/or 
when message is read by the recipient. 

However the above combined teachings do not explicitly teach where an installation 
option that identifies an installation program that can be used to install one or more modules 
allowing the content and corresponding media content to be rendered. 

Russell teaches a system/method related to for capturing, categorizing, and organizing by 
categories by using temporal segment identifiers of the media content, managing and retrieving 
multimedia (combined integrated media: audio, video text, graphics, etc. information) (see 
Russell: col 1/line 22-col 2/line 36, teaching means for annotating text to voice media content, 
col 3/lines 3-26, col 5/line 57-col 6/line 36, 45-col 7/line 16, identifiers of temporal segments of 
the multimedia content that are after the beginning of the multimedia content: col 10/line 60-col 
1 1/line 34, col 1 1/line 65-col 1 2/line 16, col 13/lines 3-31, 57-64), disclosing installation option 
that identifies an installation program that can be used to install one or more modules allowing 
the content and corresponding media content to be rendered (col 16/lines 47-col 19/line 35). 

It would have been obvious to one ordinary skilled in the art at the time the invention was 
made to modify the combined teachings as discussed with means for field via which the user can 
identify a recipient that is to receive an electronic mail notification of the new annotation, as 
discussed above, motivation would be to further enhance existing e-mail system with means for 
enabling recipient names to be resolved while user is composing an e-mail message without user 
intervention, enabling user to select from a list field adding the notification means level of 
urgency of the received message. Further modify existing system with means for communicating 
said electronic mail message includes a new annotation corresponding to media content created 
by a user and an identifier of the media content but does not include the media content, wherein 
identifier of the media content to which the new annotation corresponds includes a user- 
selectable link to said media content, as taught by Birrel, motivation would be enable a plurality 
of client machines to create/receive annotations within electronic mail messages to include media 
content such as Postscript documents, HTML pages of embedded MIME fragments all 
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implemented with standard existing software, annotate electronic mail messages with mutable 
identifiers for categorized storage/retrieval and access mail via low-bandwidth network 
connections by generating user-linkable annotations within an electronic mail message, as taught 
by Birrel; Further modifying existing system with means for having an installation option that 
identifies an installation program that can be used to install one or more modules allowing the 
content and corresponding media content to be rendered, as taught by Russell, motivation would 
be enhance existing annotation/playback-based system with of replaying annotated multimedia 
content in a significantly order that which the media was originally stored, where temporal 
markers enable the support the enhancement of a playback that will require less time to render 
media content and enables the user to navigate through the media content by using visual cues. 

Regarding claim 8, the combined teachings as discussed above, including in the electronic mail 
message, an indication of one of a plurality of annotation sets, the one annotation set 
corresponding to the media content of the new annotation; (Hou: col 3/lines 8-13, col 2/lines 25- 
37, col 3/lines 9-13, col 6/lines 26-51, col 6/line 61-66, col 7/lines 1-3, col 8/lines 61-63, col 
Mines 9-14, col 1/lines 39-44, sending/displaying mail: col 11/lines 22-25, col 6/lines 14-16, col 
1 1/lines 3-25, Fig. 1, wherein the annotations sets corresponds to categories for annotations; 
different types of media, static annotation means allows the user to add drawings and text to be 
sent via email, allows the user to record/playback annotations, an use can create reports by 
including multiple media data; col 2/line 50-col 3/line 13); 

Regarding claim 9, the combined teachings as discussed above, further comprising: allowing the 
user to identify, via the field of the user interface, a plurality of recipients to receive the 
electronic mail notification, as discussed above, and forwarding, to each of the plurality of 
recipients, the electronic mail notification of the new annotation (Hou: col 4/lines 26-3 1, Fig. 4). 

Regarding claim 10, the combined teachings as discussed above, further comprising: allowing 
the user to input, via the field of the user interface, a group identifier, as discussed above, Hou: 
col 4/lines 26-31); identifying an electronic mail address for each of a plurality of recipients 
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col 7/lines 4-51, col 2/lines 52-61, col 6/lines 10-14, col 7/line 4/line 51); wherein access to the 
media content comprises particularly receiving a user selection of an identifier, within the 
electronic mail notification, of the media content; (Birrel: composing/accessing an email 
message, col 1/lines 38-50, composing email means, col 4/lines 31-50, includes a new annotation 
corresponding to media content, multimedia content, col 4/lines 59-61, and an identifier of the 
media content but does not include the media content; col 1 2/lines 2-27 used to retrieve 
multimedia content at location associated with the identifier upon user selection of the media 
content associated with said annotation, wherein the said annotation does not include the media 
content created by the user, col 12/lines 59-col 13/line 13); 

Regarding claim 19, the combined teachings as discussed above, wherein the presenting 
comprises: displaying annotation content for the new annotation to the user; and displaying at 
least one identifier to the user, the at least one identifier including one or more of a content 
identifier that identifies the media content; a range identifier that identifies a temporal range of 
the segment corresponding to the new annotation (Sidana: in the electronic mail message, an 
identifier of the new annotation, col 8/lines 5-12, an identifier of the temporal range of media 
content Hou: col 8/lines 61-63, col 9/lines 1-4, col 6/lines 61-66, col 7/lines 1-3), an annotation 
identifier that identifies the new annotation; and an annotation set identifier that identifies one or 
more annotation sets that the new annotation is part of (Hou: col 5/lines 10-28); wherein said 
identifier of a temporal segment of the multimedia content is after the beginning of the 
multimedia content; Hou teaches means for identifying temporal ranges or segment of 
multimedia content with which the new annotation is associated, such as time temporal range 
denoting the a segment of time (i.e. beginning/end), temporal range of information denoting the 
content, range, category type, beginning and end of a segmented multimedia content supporting 
means to store, retrieve, render multimedia content (e.g. time, object identifier, begin/end of 
objects), annotation sets corresponding to the multimedia categories (e.g. image, text, audio), that 
the new or stored annotation is part of (Figs. 6, 14-15, col 7/lines 4-51, col 2/lines 52-61, col 
6/lines 10-14, col 7/line 4/line 51). 
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Regarding claim 22, the combined teachings discussed above, wherein the receiving a user input 
comprises receiving a user selection of an identifier of the annotation (Hou: col 6/lines 61-66, col 
7/iine 1-3). 

Regarding claim 24, the combined teachings as discussed above, one or more computer-readable 
media as recited in claim 23, wherein the user-selectable identifier comprises a uniform resource 
locator (URL) that identifies a server and a location at the server where the media content is 
located (Sidana: col 7/lines 62-col 8/line 26, Birrel, location-URL, col 3/lines 44-55, links in 
email, col 12/lines 22-67, not the multimedia content, col 1 3/lines 1-7) 

Regarding claim 33, the combined teachings as discussed above, teach means/method 
comprising; receiving an electronic mail thread including a plurality of electronic mail messages 
(Pizano, col 2/line 32-60 organizing received mail thread); creating a plurality of annotation from 
at least one of the electronic mail messages in the electronic mail thread (Hou: generating an 
electronic mail message including both the content of the new annotation and an identifier of the 
media content, create electronic mail messages col 3/lines 8-13, create means: col 2/lines 25-37, 
col 3/lines 9-13); and adding a plurality of annotation to an annotation database (Hou: add the 
new annotations, col 9/lines 15-32, col 1/lines 17-20, adding annotations means: col 5/lines 29- 
52, report: col 6/lines 10-14, Fig. 3, saving in annotation database). 

Regarding claim 35, the combined teachings as discussed above, wherein: the creating comprises 
generating, for each of the plurality of electronic mail messages, an annotation (Hou: generating 
an electronic mail message including both the content of the new annotation and an identifier of 
the media content, create electronic mail messages col 3/lines 8-13, create means: col 2/lines 25- 
37, col 3/lines 9-13); and the adding comprises adding each of the generated annotations to the 
annotation database (Hou: add the new annotations, col 9/lines 15-32, col 1/lines 17-20, adding 
annotations means: col 5/lines 29-52, report: col 6/lines 10-14, Fig. 3, saving in annotation 
database). 
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Regarding claim 36, the combined teachings as discussed above, further comprising locating, in 
the electronic mail thread, an identifier of media content that the annotation corresponds to 
(Sidana: mail message further includes a unique identifier of the new annotation, col 7/lines 62- 
col 8/line 26, wherein the electronic mail message further includes an identifier of one or more 
annotation sets that the new annotation is associated with, Hou: col 5/lines 10-28). 

Regarding claims 40-42, the combined teachings as discussed above, further teach a 
system/method for receiving an electronic mail message including data corresponding to media 
content that is not included in the mail message (Birrel; electronic mail message includes an 
annotation corresponding to the media content col 12/lines 2-27, including user-selectable link 
to said media content wherein the said annotation does not include the media content within 
said mail message, col 12/lines 59-col 13/line 13); locating the data in the electronic mail 
message; generating, after locating the data, a new annotation corresponding to the media 
content, (Sidana; abstract, col 2/lines 34-59, col 4/lines 14-33, col 8/lines 5-12, Pizano; col 
1/lines 60-67, col 2/lines 32-60, col 1/lines 60-67, col 3/lines 32-47, col 5/lines 26-36); wherein 
the new annotation corresponding to media content data included in electronic mail message 
includes; an field including data identifying a sender of the electronic mail message as an author 
of the new annotation corresponding to the media content included in the electronic mail 
message (Birrel, col 1 1/lines 53-67 and a field including data identifying the time at which the 
new annotation is generated, a field including data identifying a title of the new annotation 
(Birrel, col 12/lines 12-14, a field including the located data (Birrel, col 12/lines 1-11, and a 
field including data identifying the media content to which the new annotation corresponds 
(Birrel, col 12/lines 22-44); (Pizano, Fig. 6, fields time, sender, annotations comprising media 
content, annotation comprising sender, time fields, col 5/lines 56-col 6/line 21); an installation 
option that identifies an installation program that can be used to install one module allowing the 
content and corresponding media to be rendered (Russell; installation option that identifies an 
installation program that can be used to install one or more modules allowing the content and 
corresponding media content to be rendered, col 16/lines 47-col 19/line 35); wherein installation 
option comprises an user-selectable link that identifies the installation program, wherein 
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installation option comprising means for loading/executing software that allows the content and 
corresponding media content to be rendered (Birrel; col 12/lines 28-44); 

Response to Arguments 

6. Regarding claim 1, applicant argues (A): prior art of record Hou does not teach claim 
limitation as amended, asserting Hou teaches sending a report including annotation to a receiver, 
and making a new annotation which is reply to a selected segment of another annotation, 
however does not teach as amended limitation recites: "generate new annotation based on 
electronic mail messages received both in response to the sent electronic mail messages and not 
in response to the sent electronic mail messages". 

7. Regarding claim 2, applicant argues (B): prior art of record Sidana does not teach claim 
limitation, i.e. "a media server to manage streaming the multimedia content to the client 
computer", asserting that the disclosure of an HTML web document transferable between a client 
and server where the retrieved document is displayable by the client browser on the screen does 
not suggest a media server streaming multimedia content to the client; where streaming is to be 
interpret as meaning data representing various media types being provided over a network to a 
client computer on a real-time, as-needed bases, rather than pre-delivered in its entirety before 
playback, making reference to specification (page 2, lines 14-18). 

8. Regarding claims 5, 13, 25, 28, 31, 39, 6, 17, 19, and 23, presented on pages 13-21, are 
all arguments directed to claim limitations as amended, wherein amended limitations have above 
modified above and addressed accordingly. 

9. Regarding claim 33, applicant argues, (C) prior art of record Hou, Sidana nor Goodhand 
teach claim limitation as amended: "creating a plurality of annotations from the plurality of 
electronic mail messages in the electronic mail thread"; 
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corresponding to the group identifier; and forwarding, to each of the plurality of recipients (Hou: 
col 4/lines 26-3 1, Fig. 4), the electronic mail notification of the new annotation. 

Regarding claims 11-12, the combined teachings as discussed above, further comprising: 
presenting, as part of the user interface, a default set of recipients to receive the electronic mail 
notification, as discussed above, Hou: col 4/lines 26-3 1). 

Regarding claims 17-18, 20-21, and 23, the combined teachings as discussed above, teach means 
for receiving an electronic mail notification of a new annotation corresponding to media content, 
the media content having a plurality of temporal segments (Hou: col 6/line 61-66, col 7/lines 1-3, 
col 8/lines 61-63, col 9/lines 9-14) of media content col 1/lines 39-44, col 3/lines 9-13, receiving 
an electronic mail message including both the content of the new annotation and an identifier of 
the media content create electronic mail messages col 3/lines 8-13, col 2/lines 25-37, containing 
media content identifiers/temporal range markers col 6/line 61-66, col 7/lines 1-3, col 8/lines 61- 
63, col 9/lines 9-14, col 1/lines 39-44); displaying/presenting the electronic mail notification to a 
user (Hou: display annotations means, col 8/lines 43-48, sending and displaying mail: col 
1 1/lines 22-25); receiving a user selection to access the plurality of media content; and 
automatically accessing, in response to the user input selection of the identifier of the media 
content, a media server to stream one of the plurality of segments that corresponds to the new 
annotation to the user (Sidana: displaying means ; col 4/lines 14-43, accessing via user input: col 
5/lines 58-col 6/line 6, col 7/lines 62-col 8/line 26, wherein user input comprises the selection of 
a URL of the media content identifier), wherein said identifier of a temporal segment of the 
multimedia content is after the beginning of the multimedia content is associated with; Hou 
teaches means for identifying temporal ranges or segment of multimedia content with which the 
new annotation is associated, such as time temporal range denoting the a segment of time (i.e. 
beginning/end), temporal range of information denoting the content, range, category type, 
beginning and end of a segmented multimedia content to support means to store, retrieve, render 
multimedia content (e.g. load/display, time, object identifier), move, time, object, at the 
begin/end of other objects), annotation sets corresponding to the multimedia categories (e.g. 
image, text, audio), that the new/stored annotation multimedia content is part of (Figs. 6, 14-15, 
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10. In response to point (A): Prior art of record Hou teaches generate a new annotation based 
on electronic mail messages received both in response to the sent electronic mail message and 
not in response to the sent electronic mail messages: (see Hou; col 2/lines 25-30, col 50-col 
3/line 13, col 3/line 65-col 4/line 14, col 4/lines 26-31, col 6/lines 14-16). 

11. In response to point (B): (i) Prior art of record Sidana teaches a media server configured 
to manage streaming the multimedia content to the client computer (Sidana: abstract, col 2/lines 
34-59, col 4/lines 14-33, Fig. 1, element 106, 110, 130, annotation server 120, col 2/lines 41-59, 
col 4/lines 20-31, where an identifier, identifiers the media content associated with new 
annotation content, col 8/lines 5-12), (ii) In response to applicant's argument that the references 
fail to show certain features of applicant's invention, it is noted that the features upon which 
applicant relies (i.e., "a media server to manage streaming the multimedia content to the client 
computer, where streaming is to be interpret as meaning data representing various media types 
being provided over a network to a client computer on a real-time, as-needed bases, rather than 
pre-delivered in its entirety before playback"), are not recited in the rejected claim(s). Although 
the claims are interpreted in light of the specification, limitations from the specification are not 
read into the claims. See In re Van Geuns, 988 F.2d 1 181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

12. In response to point (C): (i) It is respectfully noted that according to applicant's 
specification, email application used as the basis of claimed invention is a convention e-mail 
application, having typical user-selectable features (Figs. 8-9, (e.g. 300, 316, 318)), such as 
viewing, creating, manipulating (e.g. replying, creating a new), wherein typical email 
applications provide a user to a forward option that allows the user to send a received email 
message to one or more other users, optionally appending an additional message by the user, 
such email messages can be forward from user to user, creating a string of mail messages often 
referred to as an "email thread", wherein such forwarding can be found in numerous settings, 
such as designers that carry on "conversations" or discuss various options with one another (see 
page 26-34, line 5); (ii) claim recites receiving an electronic mail thread including a plurality of 
electronic mail messages', (Official Notice has been taken as a well Known feature(s) in the art); 
creating a plurality of annotation (Hou: col 1/lines 6-10, 37-44, col 2/lines 31-37, 50-61, col 
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6/lines 26-37) from the plurality of electronic mail messages in the electronic mail thread (Hou: 
generating an electronic mail message including both the content of the new annotation and an 
identifier of the media content, create electronic mail messages col 3/lines 8-13, create means: 
col 2/lines 25-37, col 3/lines 9-13, Official Notice has been taken as a well Known feature(s) in 
the art; (i.e. creation/reception group of messages and replies related by topic, means for 
organizing received mail thread, forward/receiving to/from multiple users means); and adding a 
plurality of annotation to an annotation database (Hou: add the new annotations, col 9/lines 15- 
32, col 1/lines 17-20, adding annotations means: col 5/lines 29-52, report: col 6/lines 10-14, Fig. 
3, saving in annotation database). 

13. Applicant's presented arguments, filed 11/01/00 have been fully considered but are not 
deemed to be persuasive. 
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Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Prieto, B. whose telephone number is (703) 305-0750. The 
Examiner can normally be reached on Monday-Friday from 6:30 to 4:00 p.m. If attempts to 
reach the examiner by telephone are unsuccessful, the Examiner's Supervisor, Mark H. 
Rinehart can be reached on (703) 305-4815. The fax phone number for the organization where 
this application or proceeding is assigned is (703) 308-6606. Any inquiry of a general nature or 
relating to the status of this application or proceeding should be directed to the receptionist 
whose telephone number is (703) 305-3800/4700. 

Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

or faxed to: 

(703) 308-9051, (for formal communications intended for entry) 

Or: 

(703) 305-7201 (for informal or draft communications, please label 
"PROPOSED" or "DRAFT") 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington. VA., Fourth Floor (Receptionist), further ensuring that a receipt is provided stamped 

<C TC2100'\ 

B. Prieto 
Patent Examiner 

July 15, 2001 
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